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Abstract 



The Bernoulli sieve is a version of the classical balls-in-boxes occu- 
pancy scheme, in which random frequencies of infinitely many boxes 
are produced by a multiplicative random walk, also known as the resid- 
ual allocation model or stick-breaking. We give an overview of the limit 
C^ , theorems concerning the number of boxes occupied by some balls out 

of the first n balls thrown, and present some new results concerning 
the number of empty boxes within the occupancy range. 



^^ ■ 1 Introduction 

O 

r^ ■ In a classical occupancy scheme n balls are thrown independently in an 

infinite array of boxes with probability pk of hitting box k = 1,2,..., where 

Jjf^ I {pk)k£N is a fixed sequence of positive frequencies summing up to one. The 

(<— ^ ' quantities of traditional interest are 

• Kn the number of boxes occupied by at least one of n balls, 

K> , • Kn^r the number of boxes occupied by exactly r out of n balls, 

5r ! • ^n the range of occupancy, equal to the maximal index of occupied 

box, 

• L„ := M„ — Kn the number of empty boxes within the occupancy 
range. 
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• Zn the number of balls in the M„th box. 

In applications 'boxes' are clusters, species, types of data etc. The quantities 
in the list characterise the sample variability, which for large n is dominantly 
determined by the boxes occupied by a few balls, thus determined by the 
way the frequencies p^ approach zero as A; — ?> oo. The first two variables are 
functionals of the induced partition of n, defined as the unordered collection 
of positive occupancy counts. 

The Bernoulli sieve is a version of the occupancy scheme with random 
frequencies 

Vk--=WxW2---Wu^r(\-Wk). A:GN, (1) 

where (WA;)fceN are independent copies of a random variable W taking values 
in (0, 1). The name derives from the following recursive construction based 
on i.i.d. q\^ =d 1 — W: at round 1 a coin with probability qi for heads is 
flipped for each of n balls and every time it turns heads the ball is put in 
box 1, then at round 2 a coin with probability 52 for heads is flipped for 
each of the remaining balls and every time it turns heads the ball is sent to 
box 2, and so on until all balls are allocated in boxes. 

It is useful to identify frequencies ([I]) with the lengths of component in- 
tervals induced by splitting [0, 1] at points visited by a multiplicative random 
walk {Qk)kmoi where 

i 
Qo:=l, Qj-=]\W,, jGN. 

In the spirit of Kingman's 'paintbox representation' of exchangeable parti- 
tions p8], we may identify the boxes with open intervals {Qk^Qk~i)^ and 
mark the balls by independent points Ui, . . . ,Un sampled from the uniform 
[0,1] distribution, independently of (Qk)- The event Ui € {Qk-i,Qk) then 
means that ball i falls in box k. Keep in mind that in the natural order the 
intervals are indexed from the right to the left, thus the occupancy range is 
determined by the interval containing the leftmost mark min(C7i, . . . , C7„). 

The Bernoulli sieve has nonrandom frequencies only when the law of W 
is a Dirac mass 6p located at some p G (0,1), the frequencies pk comprise 
then a geometric distribution. Results for this case can be readily recast 
from the numerous studies on sampling from the geometric distribution [5l 
m [iHl [25] and related models like the leader election algorithms [H [HI |22l [28] , 
absorption sampling [71 [23] etc. It is known that asymptotic expansions of 
the moments of Kn,Mn and many other quantities have a component that 
oscillates periodically on the logn-scale with a small amplitude [HI [27] . 



The same applies to distributions of the L„'s |191 I26j . There are some 
pecuharities in the symmetric case p = 1/2 |lll 128] , 

The best analytically tractable case involves random factors having heta{9, 1) 
density ¥{W € dx} = Bx^^^dx on (0, 1) with parameter ^ > 0. In this case 
the Bernoulli sieve may be viewed as a way to generate a random partition 
of n which follows the multivariate distribution known as the Ewens sam- 
pling formula \T\. This model has been widely studied in connection with 
problems of combinatorics, statistics and biology. In particular, the case 
6 = 1 of uniform factors is related to records and cycle patterns of random 
permutations under the uniform distribution on the symmetric group. It 
is well known [I] that {Kn — 9logn)/{6logny''^ is asymptotically normal, 
and that the i^n,r's converge jointly to independent Poisson(^/r) random 
variables. These classical results are complemented by the observation that 
Mn exhibits the same asymptotics of moments and distribution as Kn , and 
the number of empty boxes has the following surprising limit law: 

Theorem 1.1. [16] // W has beta(^, 1) distribution then Ln — >d L^o, where 
Loo has probability generating function 

' r{i + 2e -9s) ' *^^^'^J' 

which corresponds to a mixed Poisson distribution with the parameter dis- 
tributed like 9 I log(l — W)\. 

Throughout we shall use the following notation for the moments 

fi:=E\logW\, cj^ :=Var(logTy), u :=E\log{l -W)\, 

which may be finite or infinite. The standing assumption for what follows 
is that the distribution of |logVK| is non-lattice. In particular, the case of 
sampling from the geometric distribution will be excluded. 

2 Markov chains and distributional recursions 

A random combinatorial structure which captures the occupancy of boxes by 
n indistinguishable balls is the weak composition C* comprised of nonnega- 
tive integer parts summing up to n. The term weak composition means that 
zero parts are allowed, for instance, the sequence (2, 3, 0, 1, 0, 0, 1, 0, 0, 0, . . .) 
(padded by infinitely many O's) is a possible value of C^. A related structure 
which contains less information is a composition Cn obtained by discarding 



zero parts of C*. Discarding further the order of parts in Cn yields a ran- 
dom partition of n. The parts of C* can be represented (see pSj, p. 452]) 
as the magnitudes of jumps of a time- homogeneous nonincreasing Markov 
chain Q* = {Q'^{k))k£No on integers, which starts at n and moves from n to 
m with transition probabihties 

q*(n,m)= ( "^^Eil - WV-'^W"' , m = 0,...,n. 

In the same direction, parts of the composition C„ are the magnitudes of 
jumps of a Markov chain Qn = {Qn{k))keNo with transition probabihties 

q{n,m)=[J T^EWn ' m = 0,...,n-l. 

This Markovian reahsation imphes the following distributional recursions 
(see [m Section 3]): 

Mo = 0, M„ =d Mq,(i) + 1, nGN, 



0, 


M„ 


0, 


Kn 


Ln 


=d 



Ko = 0, Kn =d Kg^ii) + I, nGN, (2) 

Lo = 0, L„, =d Lq*(i) + 1|Q* (!)=„}, n G N, (3) 

where in the right-hand side Qni^) is assumed independent of {Mn : n £ N} 
and {Ln : n € N}, and (5n(l) independent of {Kn : n G N}. Analysis of the 
recursions by known direct methods is difficult, as these impose restrictive 
conditions on the moments of (5n(l) or (5*(1). Nevertheless, coupling with 
the multiplicative random walk allows to gain a lot of information about the 
compositions. For instance, let g{n, m) be the potential function, equal to 
the probability that Qn ever visits state m, 

g{n,m) = ^P{(5„(j) = m}. 

3=0 

The coupling implies that i \\.2\ Proposition 5]) 

lim g{n,m) = , (4) 

n-yoo nm 

which is if ;U = oo. 

The coupling readily implies stochastic subadditivity Mn+m <d Mn + 
-M^ where the terms in the right-hand side are independent. Indeed, note 
first that M„ is nondecreasing. Now, when n balls have been allocated 



within the range M„, adding m new balls leads to (stochastically) niaximal 
increase of the occupancy range when all m fall outside the old range Mn, 
in which event the new range of occupancy is distributed like M„ + M^. 
With analogous notation, Ln+m <d Ln + L'^ for exactly the same reason 
(although Ln is not monotone). 

3 Asymptotics of M„ 

Passing from the multiplicative to conventional (additive) random walk we 
introduce 

So:=0, Sk:=\logWi\ + ... + \logWk\, A: G N. (5) 

In this scenario the Bernoulli sieve can be defined as allocation of balls 
with exponentially distributed marks Ej = —logUj, 1 < j < n, in boxes 
{Sk,Sk+i), k G No- Define 

Nt := mf{k > 1 : S'fe > i}, i > 0, (6) 

which is the first time (Sk) enters (t,oo). From the extreme- value theory 
we know that the maximum statistic T„ := max(£'i, . . . , En)) satisfies T„ — 
logn — >rf T, where T has the standard Gumbel distribution P{T < x} = 
exp(— e~^), X € M. A key observation is that 

thus the asymptotic behaviour of Af„ is very much the same as that of 
Mogm and the latter can be concluded by means of the renewal theory. A 
complete description of possible limit laws and scaling/centering constants 
for the number of renewals Nt [16, Proposition A.l] leads to the following 
classification of possible limit laws for M„. 

Theorem 3.1. [16j The following assertions are equivalent: 

(i) There exist sequences {a„,6„ : n € N} with a^ > and 6„ € M such 
that, as n ^- oo, the variable {Mn — hn)/an converges weakly to some 
non- degenerate and proper distribution. 

(ii) The distribution of \\ogW\ either belongs to the domain of attraction 
of a stable law, or the function P{| log VF| > x} slowly varies at oo. 

Accordingly, there are five possible modes of convergence: 



(a) If (7^ < oo then, with constants bn = fJ- ^logn and an = (^ ^cr^ log n)^'^, 
the limiting distribution of {Mn — bn)/an is standard normal. 

(b) If a^ = oo, and 

/ y'^F{\logW\edy} ~ L{x) j; ^ oo, 
Jo 

for some function L slowly varying at oo, then, with bn = ^^^logn 
and On = Ai~^'^cnog„i, where c{x) is any positive function satisfying 
liuix-^oo xL{c{x))/(?{x) = 1, the limiting distribution of{Mn — bn)/an 
is standard normal. 

(c) // 

F{\logW\>x} ~ x^"L(x), x^oo, (7) 

for some L slowly varying at oo and a G (1,2) then, with bn = 
fi~^logn and an = fJ'~^'^~^^''°'ciogn, where c{x) is any positive func- 
tion satisfying lim^_j.oo xL(c(x))/c"(x) = 1, the limiting distribution 
of {Mn — bn)/an is a-stable with characteristic function 

tH>exp{- 1*1^(1 -a)(cos(7ra/2) + ism(7ra/2) sgn(t))}, t £ R. 

(d) Assume that the relation ([7]) holds with a = 1. Let r : M — )■ M be any 
nondecreasing function such that lim^^-^oo xF{\ log W\ > r{x)} = 1 and 
set 

m{x) := / F{\\ogW\> y}dy, x > 0. 
Jo 

Then, with bn = ^ogn/(m{logn/r{m(\ogn)))) and 

r(log n/m(log n)) 
m(iognj 

the limiting distribution of {Mn — bn)/an is 1-stable with characteristic 
function 

t h^exp{-|t|(7r/2-ilog|t|sgn(t))}, t G M. 

(e) If the relation ([7]) holds for a G [0, 1) then, with 6„ = and a„ := 
log"n/L(logn), the limiting distribution ofMn/un is the Mittag-Leffler 
law 6a with moments 

°° k^ 

x^ eJdx) = -^, -— -, A; G N. 

^ ' r^(l-a)r(l + a/c) 



4 Asymptotics of Kn 

Loosely speaking, z^ controls the mean number of empty boxes, so that 
1/ < oo implies lim„_..ooEL„ = v/fi < oo (Theorem 17.11 to follow). Thus 
when 1/ < CX3 the identity Kn = Mn — Ln suggests that K^ does not differ 
much from M„. A first result of this kind was obtained in |12j : assuming 
z/ < CO and u'^ < oo it was shown that 



{Kn — ^i ^ logn)/-\/cj^/i '^ log n ^'d normal (0, 1), n — > oo. 

The proof was based on a careful analysis of the recursion ^ to conclude 
on the asymptotics of Var i^„ and to eventually prove the normal limit. 

The similarity between M„ and Kn was justified in full generality in [16] , 
where is was shown that under the assumption u < co Theorem 13.11 remains 
valid if Mn is replaced by Kn ■ 

Another approach which allows one to treat the cases of finite and infinite 
z/ in a unified way was proposed in p!5]. It was suggested to approximate 
Kn by N*{logn), where 

N*{x) := #{k e N : pk > e-""} 

= #{keN:Wi---Wk^iil-Wk)>e~''}, x > 0. 

The connection exemplifies the general idea that the variability of Kn stems 
from randomness in frequencies (pk) superposed with randomness in sam- 
pling, and the first often plays a dominating role through the conditional 
law of large numbers Kn ^ E(i^„ | (pk)) a-s. (see [23]). Thus we believe that 
the approach based on N*{x) offers a natural and the most adequate way 
to study the asymptotics of Kn- The following result was proved in [15j . 



Theorem 4.1. If there exist functions f : M+ — ?• M+ and g : M+ -^ M such 
that {Nt — g{t))/f{t) converges weakly (as t — >■ ooj to some non- degenerate 
and proper distribution, then also {Kn — hn)/an converge weakly (as n — ?> co) 
to the same distribution, where the constants are given by 

Aogn 

bn= g{logn-y)F{\log{l-W)\£dy}, a„ = /(logn). 

Jo 

As in [16j, the convergence criterion for Nt leads to a complete charac- 
terisation of possible normalisations and limiting laws for Kn, see Corollary 
1.1 in [15j. But Theorem 14.11 savs more: if zv = oo the behaviour of L„ may 
affect the asymptotics of Kn = Mn — Ln- The following example illustrates 
the phenomenon. 



Example 4.2. Assume that, for some 7 G (0, 1/2), 

nW>x}= ] — , xG[0,l). 

1 + llog(l -x)P 

Then 



Elog^W<oo and F{\log{l - W)\ > x} r^ x^'^ as x ^ 00, 

and in this case, 

a„ = const log ' n and dn = f^^ (logn— (1— 7)^ log ~'''n + o(log ^'^ n)). 

Thus we see that the second term dn — fi^^logn of centering cannot be 
ignored. Moreover, one can check that 

ELn ~ - > , —, — -^ On- fi log n ~ — log ' n, 

^^ A; ^(1-7) 

which reveals the indispensable contribution of L„. 

5 Weak convergence of K^^r 

Assume /J: < 00. For B := {Yl^^i Wi : k £ No} the set of sites visited by 
the multiplicative random walk, consider a point process with unit atoms 
located at points of — logi? (which are the sites visited by Sk, k G No). 
By the renewal theorem the point process — log B — log n vaguely converges 
to a shift-invariant renewal process V on the whole line. Therefore, the 
point process nB converges vaguely to a point process B := exp(— P) on 
IR+. Think of intervals between consequitive points of ^B as a series of boxes. 
Note that the process is self-similar, meaning that cB =d B for every c > 0, 
and has the intensity measure {ixx)~^(ix, so the atoms accumulate at 
and 00. In the role of balls assume the points of a unit Poisson process U 
independent of S. A well-known fact of extreme value theory is that U is the 
vague limit of the point process with unit atoms located at nUj, 1 < j < n. 
The location of the leftmost atom ofU, say Y, has exponential distribution. 
For r > define Kr to be the number of component intervals of {Y, 00) \ B 
that contain exactly r atoms of U. The existence of weak limits for the 
occupancy counts is read off from the convergence of point processes: 



Theorem 5.1. [TTj As n —^ oo we have the joint convergence in distribution 

along with 

EKnr ^Ei:,. = — , r > 0. 
r/i 

When W =d beta(^, 1) the process B is Poisson with intensity Ox^^dx. 
By self-similarity, the partition induced by allocation of n leftmost atoms 
of hi is the Ewens partition. The theorem allows to re-prove the results 
on asymptotics of the Ewens partition mentioned in Introduction, along 
with Theorem 11.11 Except the heta{9, 1) case no explicit formulas for the 
distribution of occupancy counts are known; in general the Kr^s are neither 
independent, nor Poisson. See more on self-similar partitions in [131 Section 
5]. 

6 Asymptotics of Zn 

The variable Zn is analogous to the number of winners in the leader election 
algorithm H El [Q [25] . 



Theorem 6.1. [16] The number of balls in the last occupied box satisfies: 
(1) If fi < oo then Zn -^d Z, n ^ oo, where the variable Z has distribution 

E(l-W)'' 

P{Z = k} = -^ ; '-, k^N. 

Ilk 



(2) // dZI) holds with a € [0, 1) then 

-^d -^"■"^ n — > oo. 



logZ, > ^H 



logn 

where the law of Z^'^' is 5i, while for a € (0,1) we have Z^"^' =d 
beta(l — a,a). 

(3) // d?]) holds with a = 1 and fj, = oo, then 

— ^-^dZ'^', n ^ oo, 

m(iognj 

where m{x) = f^ P{[ log VF[ > y}dy , and Z^'^' =d uniform[0, 1]. 



The case // < oo is quite elementary, as is seen from 

F{Zn = m} = g{n, m)P{Q^(l) = 0} = g{n, m)^—^^ (8) 

and (j3]). In the case // = cx) the result follows from the known limit distribu- 
tion of the undershoot U{z) = z—S]\f(^z)-i (see |8l[T0]) and the representation 

F{Zn >k}= F{U{En,n) > ^n,n " ^n-fc,n}, ^ € N, 

where Ei^n < • • • < En,n = Tn are the order statistics of the exponential 
variables Ej, 1 < j < n. 

7 Asymptotics of Ln 

Although there is an explicit formula 

'r,\ 1 _ F.n - U/\'' 

(9) 



EL„=V(-ir'f"V-'^"-f'' 

fe=i ^ ^ 



it does not seem possible to employ it in order to conclude on the asymptotic 
behaviour of EL„ without restrictive additional assumptions. 
Using a different approach we arrived at 

Theorem 7.1. The expectation EL„ exhibits the following asymptotic be- 
haviour: 

(i) If jjL = oo and i^ = oo then 

EW Ely" 

lim inf — < lim inf EL„ < lim sup EL„, < lim sup 



n^oo E(l - W)"- - n->oo " „_^oo ~ n^oo E(l - W)' 

In particular, 

EW 
hm — TTT^ = 70 G 0, oo 

n^oo E(l - W)"^ 'U I , J 

implies lim„_i.oo EL„ = 70 . 
(ii) If u < 00 and jjl < 00 then 

lim ELn = V I [1. 

n— >oo 

(iii) If n < 00 and i/ = 00 then, as n ^ 00, 

ELn ~ - / dy. 

i^Ji y 
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Proof. Part (i). Set Sm = e(i-w)"^ • ^^ ^^^^ ^^^ *^^ representation 

EL„ = Esz„, (10) 

which follows from ([8]). The array Cn,m '■= ^{Zn = w-} verifies the conditions 
of Lemma |8. II in Appendix, in particular by the assumption fj, = oo. Hence 
the lemma can be applied to tn = EL„, whence the assertion. When 70 is 
well defined the proof is simpler, as in this case the statement follows from 
pOj) . divergence of Zn, and by using dominated convergence in the case 
7o < 00, respectively using Fatou's lemma in the case 70 = 00. 

See HU and [17J for (ii). 

For part (iii) we use the poissonised version of the Bernoulli sieve, in 
which balls are thrown one-by-one at the epochs of a unit Poisson process 
(Ilt)t>o, independent of W^s. One can check that 

00 
E(Ln.|(T^.)..N) = E (e-*^--^-i(i-^^) - e"*^--^-^) . 
fc=i 

Recalling definitions (0),® and setting ip{t) := Ee^^^^^^), U{x) := EN^ = 
Xl^iPlSfc-i < x}, we have 

ELut = E^ (c^(te-'^'=-i)-exp(-te-'^'=-i) j 
fc=i ^ ^ 

= / ((^(te-^)-exp(-te~^')Jt/(dx) (11) 

~ ^^ > k\ {1-EWf^) ' ^ ' 

where the familiar formula for Laplace transform of the potential measure, 
00 -1 



has been utilised. Note that (J12p is an obvious counterpart of 
Set K{t) = ip{e^) — exp(— e*), t G M. Since z^ = 00 and 

r-OO -z{l~W) 



we conclude that 



/ dz=|log(l-t^)|, 

Jo z 

/ K{z)dz = 00. (13) 

J —00 



lim 

t— >oo 
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Applying a minor extension of [291 Theorem 5] to the equahty 

ELn,= / K{t-x)U{dx), (14) 

Jo 

which is equivalent to ([IT]) , yields 



ELu,^- I y^(e")dx ~ 1 r ^dx. 
fJ- Jo fJ' Jl X 

The asymptotics of EL„ is now obtained by the depoissonisation Lemma 
in Appendix. The lemma is applicable because ELn(i) is slowly vary- 
ing. Indeed, slow variation of J^ ip{u)du/u is checked straightforwardly from 
ip{t) X and the divergence of the integral for t = oo. D 

Similarly to the above, the proof of the next theorem is based on the 
poissonisation technique. 



Theorem 7.2. [16] If ^ < oo and u < oo then Ln — >d L^ as n —)• oo for 
some random, variable L^ whose distribution satisfies 

Moreover, the convergence of all moments holds, i. e. EL^ — )• EL^ < oo for 

ken. 



It is also known that if ;U < oo and u = oo then L„ -^^ oo (see [Tj 
and that L„ -^p if i/ < oo and |U = oo. In the cases not covered by these 
results the question about the weak convergence of Ln is open. 

Note that Theorem 17.21 only gives implicit specification of the limit law 
through distributions of Ln's, which are not easy to determine, with one 
remarkable exception. Obviously from the recursive construction of the 
Bernoulli sieve, the distribution of Li is geometric with parameter EVF. 
Curiously, the same is true for all n provided the law of W is symmetric 
about the midpoint 1/2. 

Proposition 7.3. IfW =d 1 — W then Ln is geometrically distributed with 
parameter 1/2 for all n G N. 
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Proof. The argument is based on the recursion ^ for marginal distributions 
of the L„'s. The symmetry W =d I - W yields EW'' = E(l - W)'' for all 
/c G N and 

P{Q:(1) =n} = P{Q:(1) = 0} (15) 

for all n € N. We will show by induction on n that P{Ln = k} = 2^ for 
all fc G No- Using ([3]) and ([l5]) we obtain 



n-l 



p{L„ = o} = p{g;(i) = o} + ^p{Lfc = o}p{Q;(i) = fc} 

fc=i 

= P{Q;(l) = 0} + ^(l-2P{Q:(l)=0})=i, 

by the induction hypothesis. Assuming now that f{Ln = i} = 2^*^^ for all 
i < k we have 

n-l 

P{L, = k} = Y, IP{Q;(1) = J}F{ii = k} + P{Q:(1) = n}P{L„ = k - 1} 
= 2-'^-! (l - 2P{Q;(1) = 0}) + P{Q:(1) = 0}2-'= = 2-''-\ 



and the proof is complete. D 

Alternatively, one can use a representation of L„ through the sojourns of 
the Markov chain Q^ in positive states. Indeed, recall that Li has geometric 
distribution with parameter MW. Then using (llSp and induction it can be 
checked that the distribution of L„ does not depend on n > 1. 

8 Appendix. 

For ease of reference we include a result due to Toeplitz and Schur (see [20] . 
Theorem 2 on p. 43 and Theorem 9 on p. 52). We rewrite it in a form 
suitable for our purposes. 

Lemma 8.1. Let {s„,n G N} be any sequence of real numbers and let 
{cnm, n,m & N} be a nonnegative array. Define another sequence {tn,n G 

N} by tn = Ylm=l CnmSm- If 

(i) lim„^oo Cnm = for all m, 
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then 



liminf Sn < liminf t„ < limsupt„ < limsupSn < +00. 

n— >oo n— ^-oo 



Now we address the issue of depoissonisation. 

Lemma 8.2. // KLjjt is slowly varying and linij_>oo lELn^ € (0, 00] then 

EL„~ELn„, as n — > 00. 

Proof. For any fixed e € (0, 1), 

ELn, = ELn,l{|n,-t|>et} + ELn,l{|n,-t|<et} =: Ait) + Bit). 

Sublinearity of ELn^ and the elementary large deviation bound for the Pois- 
son distribution [2], 

F{\Ut -t\> et} < cie-''^\ t > (16) 

with some ci, C2 > 0, yield Ait) — )• 0. 

It remains to evaluate Bit). Since both Mn and Kn are non-decreasing, 
we have 

Bit) = E(Mn, - i^n,)l{|n,-t|<et} < E^[(i-.)t] + E(M[(i+£)t] - M[{i-e)t])- 

Similarly, Bit) > EL[(i+j)j] - E(M[(i+j)j] - M[(i_j)t]). To proceed, we need 
an auxiliary lemma. 

Lemma 8.3. lime^o limt^oo IE(M[(i+g)t] - M[(i_£)t]) = 0. 

Proof. Let {Vi : i G N} be the sequence of independent exponentially dis- 
tributed random variables with KVi = 1/i. The sequence Go := 0, (G„ := 
Vi + . . . + Vn — hin))n<=N is an L2-bounded martingale with respect to the 
natural filtration, where /i(n) = Yl^=i ^1 J- This implies that G„ converges 
almost surely to some random variable G, with EG = 0. By Doob's inequal- 
ity 

E(sup |G„|)2 <oo. (17) 

nGNo 

Recalling the notation T„ = max(ii^i, . . . , En), 
Dit) := e(^M[(i+,),]-M[(i_,),]) =e(^C/(%+,),])-?7(%_,),]) 

= ^{uihiiil + e)t]) + G[(i+,)i]) - Uihiiil + e)t\)) 

+ [uihi[il + e)t]))-Uihi[il-e)t\)) 

- ^{uihiiil - e)t]) + G[(i_,)j]) - Uihilil - e)t])) ] =: EDi(i) + Z?2(t,e) - IE^3(t). 

14 



Recall that Blackwell's renewal theorem states that 

lim (u{x + y)-U{x)] = -, 
x^-oo y / /x 

where the convergence is locally uniform in y G M. From this we conclude 
that \im.t-).ooD2{t,£) =logY3|. Hence, 

lim lim D2(t,£) = 0. 

By the same argument \iint-^oo Di{t) = G/fx almost surely. Now we want 
to show that [/(sup^gj^^ \Gn\) is an integrable majorant for |Di(t)|. To this 
end, we use subadditivity and monotonicity of U: 

\D^it)\ = I)i(t)l{G[„+,„>o} + (-i^i(i))l{G[„+.),<o} 

< f^(G'[(i+£)i])l{G[(i+,),j>0} + t^(-G[(i+e)t])l{G[(i+,),]<0} = f^(|G'[(i+e)t] I) < U{ sup \Gr, 

An appeal to estimate U{x) < ax + b (with some a,b > 0) and (fT7|) 
allows us to conclude that 

EC/( sup \Gn\) < oM sup \Gn\ + b < oo. 

nGNo neNo 

Consequently, invoking the dominated convergence we show that limj_!.oo lEDi(t) = 
0. Along the same lines limt_^ooEL'3(t) = is shown. The proof is com- 
plete. D 

We are ready to finish the proof. Assume first that lim(_>,oo lELn^ = c € 
(0, cxd). Letting n — ?> oo then e — > in the inequality 

E^n„/(i_,) < A{n/il - e)) + EL„ + E(M[i±^„j - M„), (18) 

we obtain liminf„_j.oo IEL„ > c. The upper bound follows in the same way 
from the inequality 

ELn„/(,+,) > EL„ - E(M„ - M[i_,„]). (19) 



In the case limi_j.oo ELn^ = oo, divide inequalities p8|) and (|T9|) by ELn„ 
and let n go to oo keeping in mind that by slow variation lim„_j.oo ^^^ ^" = 1 
for every 6 > 0. D 
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